Avoid final field modifications in Karate instrumentation by daniel-mohedano · Pull Request #11753 · DataDog/dd-trace-java

daniel-mohedano · 2026-06-26T12:32:46Z

What Does This Do

Avoids the final field modifications in Karate instrumentation:
- Instead of overwriting the field after the retries, the original ScenarioResult is updated through Karate's public addStepResult API, and SuppressErrorAdvice now suppresses every failing step of a retried attempt (not just the first).

Motivation

Avoid failures in future JDKs due to JEP 500: Prepare to Make Final Mean Final.

Additional Notes

Running an example project on JDK26 with JAVA_TOOL_OPTIONS=--illegal-final-field-mutation=debug produced the following logs:

Final field result in class com.intuit.karate.core.ScenarioRuntime has been unreflected for mutation ...

The build containing the fix doesn't produce the logs.

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Add your completed PR to the merge queue by commenting /merge. You can also:
- Customize the commit message associated with the merge with /merge --commit-message "..."
- Remove your PR from the merge queue with /merge -c
- Skip all merge queue checks with /merge -f --reason "reason"; please use this judiciously, as some checks do not run at the PR-level (note: the PR still needs to be mergeable, this will only skip the pre-merge build)
- Get more information in this doc

Jira ticket: SDTEST-3867

cit-pr-commenter-54b7da · 2026-06-26T12:45:27Z

Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	57.06	54.33	$\color{red}{\blacktriangle}$ +2.73	55.43	$\color{red}{\blacktriangle}$ +1.63	60/192
agentEvpProxy	57.70	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T12:46:33Z

Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	36.79	36.42	$\color{red}{\blacktriangle}$ +0.37	36.42	$\color{red}{\blacktriangle}$ +0.37	30/93
agentless	36.03	35.70	$\color{red}{\blacktriangle}$ +0.33	35.70	$\color{red}{\blacktriangle}$ +0.33	30/93
agentlessCodeCoverage	43.84	43.60	$\color{red}{\blacktriangle}$ +0.24	43.60	$\color{red}{\blacktriangle}$ +0.24	30/93
agentlessLineCoverage	75.56	74.82	$\color{red}{\blacktriangle}$ +0.74	74.82	$\color{red}{\blacktriangle}$ +0.74	29/92

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T12:48:24Z

Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	87.40	86.07	$\color{red}{\blacktriangle}$ +1.33	86.07	$\color{red}{\blacktriangle}$ +1.33	30/136
agentless	81.45	81.05	$\color{red}{\blacktriangle}$ +0.40	81.05	$\color{red}{\blacktriangle}$ +0.40	29/135
agentlessCodeCoverage	95.99	95.12	$\color{red}{\blacktriangle}$ +0.87	95.12	$\color{red}{\blacktriangle}$ +0.87	29/133
agentlessLineCoverage	112.19	111.62	$\color{red}{\blacktriangle}$ +0.57	111.62	$\color{red}{\blacktriangle}$ +0.57	28/132

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

dd-octo-sts · 2026-06-26T12:53:50Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.00 s	13.93 s	[-0.3%; +1.2%] (no difference)
startup:insecure-bank:tracing:Agent	12.98 s	12.94 s	[-0.7%; +1.2%] (no difference)
startup:petclinic:appsec:Agent	16.94 s	16.80 s	[-0.2%; +1.9%] (no difference)
startup:petclinic:iast:Agent	16.88 s	16.94 s	[-1.2%; +0.5%] (no difference)
startup:petclinic:profiling:Agent	16.85 s	16.72 s	[-0.4%; +1.8%] (no difference)
startup:petclinic:sca:Agent	16.95 s	16.93 s	[-0.9%; +1.1%] (no difference)
startup:petclinic:tracing:Agent	15.97 s	16.11 s	[-2.0%; +0.3%] (no difference)

Commit: 3eaec0c7 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

cit-pr-commenter-54b7da · 2026-06-26T12:56:20Z

Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	93.12	93.23	$\color{green}{\blacktriangledown}$ -0.11	93.23	$\color{green}{\blacktriangledown}$ -0.11	30/96
agentless	87.17	89.58	$\color{green}{\blacktriangledown}$ -2.41	89.58	$\color{green}{\blacktriangledown}$ -2.41	31/96
agentlessCodeCoverage	98.36	99.00	$\color{green}{\blacktriangledown}$ -0.64	99.00	$\color{green}{\blacktriangledown}$ -0.64	29/94
agentlessLineCoverage	103.60	99.00	$\color{red}{\blacktriangle}$ +4.60	99.00	$\color{red}{\blacktriangle}$ +4.60	28/93

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T12:57:34Z

Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	12.52	12.87	$\color{green}{\blacktriangledown}$ -0.35	12.87	$\color{green}{\blacktriangledown}$ -0.35	28/95
agentless	11.44	11.88	$\color{green}{\blacktriangledown}$ -0.44	11.88	$\color{green}{\blacktriangledown}$ -0.44	27/94
agentlessCodeCoverage	14.51	14.81	$\color{green}{\blacktriangledown}$ -0.30	15.11	$\color{green}{\blacktriangledown}$ -0.60	27/94
agentlessLineCoverage	19.21	18.45	$\color{red}{\blacktriangle}$ +0.76	18.82	$\color{red}{\blacktriangle}$ +0.39	27/94

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T13:05:16Z

Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	19.03	19.20	$\color{green}{\blacktriangledown}$ -0.17	19.59	$\color{green}{\blacktriangledown}$ -0.56	32/99
agentless	19.20	19.20	$\color{gray}{\bullet}$ 0.00	19.20	$\color{gray}{\bullet}$ 0.00	31/100
agentlessCodeCoverage	21.15	22.09	$\color{green}{\blacktriangledown}$ -0.94	22.09	$\color{green}{\blacktriangledown}$ -0.94	31/97
agentlessLineCoverage	44.51	44.48	$\color{red}{\blacktriangle}$ +0.03	44.48	$\color{red}{\blacktriangle}$ +0.03	33/101

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T13:10:09Z

Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	16.15	16.36	$\color{green}{\blacktriangledown}$ -0.21	16.04	$\color{red}{\blacktriangle}$ +0.11	25/90
agentless	9.88	9.73	$\color{red}{\blacktriangle}$ +0.15	9.73	$\color{red}{\blacktriangle}$ +0.15	25/90
agentlessCodeCoverage	13.81	13.40	$\color{red}{\blacktriangle}$ +0.41	13.40	$\color{red}{\blacktriangle}$ +0.41	25/89
agentlessLineCoverage	32.54	32.95	$\color{green}{\blacktriangledown}$ -0.41	32.30	$\color{red}{\blacktriangle}$ +0.24	25/89

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-26T13:31:52Z

Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	2.32	17.38	$\color{green}{\blacktriangledown}$ -15.06	16.04	$\color{green}{\blacktriangledown}$ -13.72	29/97
agentless	24.87	19.59	$\color{red}{\blacktriangle}$ +5.28	15.11	$\color{red}{\blacktriangle}$ +9.76	28/96
agentlessCodeCoverage	87.53	87.80	$\color{green}{\blacktriangledown}$ -0.27	87.80	$\color{green}{\blacktriangledown}$ -0.27	29/96
agentlessLineCoverage	170.85	133.64	$\color{red}{\blacktriangle}$ +37.21	141.90	$\color{red}{\blacktriangle}$ +28.95	28/95

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-29T12:28:59Z

Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	14.28	10.54	$\color{red}{\blacktriangle}$ +3.74	10.33	$\color{red}{\blacktriangle}$ +3.95	29/93
agentless	11.26	10.75	$\color{red}{\blacktriangle}$ +0.51	9.73	$\color{red}{\blacktriangle}$ +1.53	29/94
agentlessCodeCoverage	19.27	17.38	$\color{red}{\blacktriangle}$ +1.89	17.73	$\color{red}{\blacktriangle}$ +1.54	28/92

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

cit-pr-commenter-54b7da · 2026-06-29T12:29:54Z

Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	21.09	21.65	$\color{green}{\blacktriangledown}$ -0.56	21.65	$\color{green}{\blacktriangledown}$ -0.56	30/143
agentless	18.37	18.82	$\color{green}{\blacktriangledown}$ -0.45	18.82	$\color{green}{\blacktriangledown}$ -0.45	29/141
agentlessCodeCoverage	19.50	19.99	$\color{green}{\blacktriangledown}$ -0.49	19.99	$\color{green}{\blacktriangledown}$ -0.49	29/141
agentlessLineCoverage	29.58	29.82	$\color{green}{\blacktriangledown}$ -0.24	29.82	$\color{green}{\blacktriangledown}$ -0.24	28/140

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3eaec0c7b3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-06-29T15:05:30Z

+    "test-failed"                | false   | [TestFailedKarate]                 | []
+    "test-retry-failed"          | false   | [TestFailedKarate]                 | [new TestFQN("[org/example/test_failed] test failed", "second scenario")]
+    "test-failed-then-succeed"   | true    | [TestFailedThenSucceedKarate]      | [new TestFQN("[org/example/test_failed_then_succeed] test failed", "flaky scenario")]
+    "test-retry-continue-on-step-failure" | true | [TestContinueOnStepFailureKarate] | [


Migrate the new retry coverage to JUnit 5

The repo instructions in AGENTS.md say, "Do not write new Groovy / Spock tests and migrate the existing one to JUnit 5 if it is written in Groovy." This new data-table case adds coverage to the existing Spock spec instead, so the change leaves the new Karate retry scenario in the framework the repo is trying to retire; please move it to a JUnit 5 test or migrate the spec.

Useful? React with 👍 / 👎.

datadog-datadog-prod-us1-2

More details

The core behavioral change — replacing getAndResetSuppressFailures() (which reset after the first call, leaving subsequent failing steps in a continueOnStepFailure attempt unsuppressed) with shouldSuppressFailures() (no reset) — is correct and necessary. Karate 1.0.0 source confirms getFailedStep() is non-null when isFailed() is true, addStepResult() correctly propagates failure state, and the SuppressErrorAdvice is not re-triggered problematically because suppressFailures is set to false by the last retry's beforeExecute for all non-quarantined scenarios.

Was this helpful? React 👍 or 👎

_{📊 Validated against 7 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 3eaec0c · What is Autotest? · Any feedback? Reach out in #autotest}

feat: avoid final field modifications in karate instrumentation

acf6e29

daniel-mohedano added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels Jun 26, 2026

daniel-mohedano added 3 commits June 29, 2026 14:06

fix: revert changes

277ad1a

feat: propagate failed results to original

eb735e2

fix: spotless

9c0449a

nit: comment

3eaec0c

daniel-mohedano marked this pull request as ready for review June 29, 2026 14:58

daniel-mohedano requested review from a team as code owners June 29, 2026 14:58

daniel-mohedano requested a review from mhlidd June 29, 2026 14:58

chatgpt-codex-connector Bot reviewed Jun 29, 2026

View reviewed changes

juan-fernandez approved these changes Jun 29, 2026

View reviewed changes

datadog-datadog-prod-us1-2 Bot reviewed Jun 29, 2026

View reviewed changes

Uh oh!

Conversation

daniel-mohedano commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sbt-scalatest

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - nebula-release-plugin

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - netflix-zuul

Uh oh!

dd-octo-sts Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - jolokia

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sonar-kotlin

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - okhttp

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - spring_boot

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - sonar-java

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - pass4s

Uh oh!

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test Environment - reactive-streams-jvm

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daniel-mohedano commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

dd-octo-sts Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 26, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 •

edited

Loading

cit-pr-commenter-54b7da Bot commented Jun 29, 2026 •

edited

Loading